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Abstract 

Let G be a group acting faithfully on a set X. The distinguishing 
number of the action of G on X, denoted Dg(X), is the smallest number 
of colors such that there exists a coloring of X where no nontrivial group 
element induces a color-preserving permutation of X. In this paper, we 
show that if G is nilpotent of class c or supersolvable of length c then G 
always acts with distinguishing number at most c + 1. We obtain that 
all metacyclic groups act with distinguishing number at most 3; these in- 
clude all groups of squarefree order. We also prove that the distinguishing 
number of the action of the general linear group GL„ (K) over a field K 
on the vector space K n is 2 if if has at least n + 1 elements. 



1 Introduction 

An action of a group G on a set A is said to be faithful if only the identity 
element of G fixes every element of A. Let G be a group acting faithfully 
on X. For r G N, an r-coloring of A is a function c: A — > {l,...,r}. A 
permutation 7r of A preserves the coloring c if c(ttx) — c(x) for all x £ X. 
A coloring is said to be distinguishing if the only element in G that induces a 
color-preserving permutation of A is the identity element. The distinguishing 
number of the action of G on A, denoted Dg(X), is the smallest r admitting a 
distinguishing r-coloring of A with respect to the action of G. If there does not 
exist a distinguishing r-coloring of A for any finite r, we say that Dq{X) = oo. 

The distinguishing number was first defined by Albertson and Collins in 2 
as a property of graphs. More specifically, the distinguishing number of a graph 
M, denoted D(M), is the smallest number of colors admitting a coloring of the 
vertices such that the only color-preserving automorphism of M is the identity; 
thus D(M) = Dp LUi fM;\(V(M)). Note that distinguishing colorings of graphs 
need not be proper colorings in the graph theoretic sense: two adjacent vertices 
may or may not have the same color. Although Albertson and Collins initially 
defined the distinguishing number solely in terms of graphs, the approach they 
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chose to take is nevertheless highly group theoretic. Given a group G, they 
define the distinguishing set of G, denoted D(G), as 

D{G) = {D(Q) G is a graph with Aut(£) = G}. 

Their results in [2] center around characterizing the distinguishing set of 
a group. For example, they show that D(S4) — {2,4}. They also prove the 
following result. 

Theorem. Corollary 3.1, Theorem 6] 

(1) IfG is Abelian then max{L>(G)} < 2. 

(2) IfG is dihedral then max{D(G)} < 3. 

In addition, the distinguishing number of several families of graphs, including 
trees, hypercubes, and generalized Petersen graphs, have been studied in [3], 0, 
[7], and HDJ. 

In |12j . Tymoczko generalizes the notion of the distinguishing number to 
group actions on sets and proves results about the distinguishing number of 
actions of the symmetric group S n . She shows that the distinguishing number of 
group actions is indeed a more general question than the distinguishing number 
of graphs. For example, she exhibits a faithful ^-action with distinguishing 
number 3, contrasting Albertson and Collins' result that D(S4) = {2,4}. This 
difference highlights the fact that not all faithful group actions arc realized as 
actions of the automorphism group of a graph on its vertex set. 

Following Tymoczko, it seems natural to expand the notion of the distin- 
guishing set of a group to include all of its possible actions, not just those arising 
from graph automorphism groups. In this generalized context, we ask the fol- 
lowing question: given a group G, what is the best upper bound we can give for 
Dq(X)7 In Section^] we give an upper bound for the maximum distinguishing 
number for a large class of groups including nilpotcnt and supcrsolvable groups. 

Theorem. If G is nilpotent of class c or supersolvable of length c then G acts 
with distinguishing number at most c + 1 . 

As a corollary, we obtain that all metacyclic groups act with distinguish- 
ing number at most 3 since they are supersolvable of length 2; these include 
all groups of squarefree order. Albertson and Collins' results for Abelian and 
dihedral groups are special cases of nilpotent groups of class 1 and metacyclic 
groups, respectively. In Section |21 we compute the distinguishing number for 
an important group action, that of the general linear group over a field K on a 
vector space over K. We show that the distinguishing number of this action is 
2 if | if | > 7i + 1 where n denotes the dimension of the vector space. 

Our definition of the distinguishing number of a group action differs from 
the one given in m that we require the action to be faithful. This apparent 
restriction does not in actuality limit the question being considered, however, for 
given a nonfaithful action of G on X, we may consider instead the faithful action 
of the quotient group G / Stab(X) on X, where Stab(X) denotes the elements of 
G that fix each x £ X . Also, in contrast to both [2] and J^jj we do not require 
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our groups and sets to be finite, simply because there seems to be no reason to 
do so. We only note that if G is an infinite group acting faithfully on a set X, 
then X must be infinite as well. 

2 The maximum distinguishing number of a group 

Given a group G, let D(G) = max {D G {X) | G acts faithfully rail} denote the 
largest distinguishing number that G admits, or D(G) = oo if some Dq(X) = 
oo. In J21> Tymoczko proves the bound Dc(X) < k if |G| < fc!, a result 
originally formulated by Albertson, Collins and Klcitman in terms of graphs. 
This result holds for any action of G on X, faithful or nonfaithful. In 0, 
Albertson and Collins show that if G is Abelian then D(G) < 2 and if G is 
dihedral then D(G) < 3. Their proof is formulated only in terms of graphs but 
also holds for group actions. In this section, we generalize these results to a 
class of groups that includes all nilpotent and supersolvable groups. 

The following lemma gives some conditions under which we may character- 
ize the maximum distinguishing number of a group. The idea to consider the 
intersection of a normal subgroup with the stabilizing subgroup of orbit repre- 
sentatives was inspired by Albertson and Collins' proof for dihedral groups in 
In what follows, we use (x) to denote the subgroup generated by a group 
element x. Also, we will denote group actions by exponentiation on the right. 
Thus, the image of an element x £ X under the action of g £ G is denoted x 9 , 
and we have (x 91 ) 92 = x {9l92) for all g x ,g 2 £ G. 

Lemma 2.1. Suppose N is a normal subgroup of G with the property that if 
rii,ri2 £ N are conjugate elements in G, then (ni) = (n^). Suppose further that 
any subgroup L ofG/N has the property that D(L) < c. Then D(G) < c + 1. 

Proof. The case G = 1 is trivial. Suppose that a nontrivial group G acts 
faithfully on a set X. Choose a set U of representatives of the orbits of G 
on X (using the Axiom of Choice if there are infinitely many orbits), and let 
H = {g £ G | u 9 = u for each u £ U} stabilize the set U pointwise. We 
claim that H n N = 1. Suppose that n £ H n N, so that n stabilizes each 
u £ U. Fix any x E X and let u £ U be the representative of the orbit 
containing x. Let g £ G satisfy u = x 9 , and let H x be the stabilizer subgroup 
of x. By assumption, n stabilizes u, so gng^ 1 stabilizes x. But the fact that 
(n) = (gng^ 1 ) implies that n £ (gng^ 1 ) < H x . Therefore, n stabilizes each 
x £ X. Since G acts faithfully, n = 1 and so H n N = 1. Applying the Second 
Isomorphism Theorem, we conclude that H = HN/N < G/N . 

Now, we know that X\U is nonempty because G is nontrivial, so consider the 
action of H on X\U. This action is faithful since the action of G on X is faithful. 
Then Dh{X \ U) < D(H) < c since H is isomorphic to a subgroup of G/N . 
Then let C: X \ U — ► {1, . . . , c} be a c-coloring of X \ U that is distinguishing 
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with respect to the action of H. Now define C : X — > {1, . . . , c + 1} as 



We claim that C is a distinguishing (c + l)-coloring of X with respect to the 
action of G. Suppose g € G preserves C". Then g must fix each orbit rep- 
resentative u £ J7, since they are the only elements of color c + 1 and lie in 
different orbits. Thus g e H . Then consider the action of g on X \ U. Since 
the restriction of C to X \ {/ is a distinguishing coloring with respect to the 
action of H, and g € H preserves this coloring, we have g = 1. Therefore, C 
is a distinguishing c + 1-coloring with respect to the action of G. We conclude 
that an arbitrary action of G has distinguishing number at most c+1, and so 



The next theorem is a consequence of Lemma I/TT1 Following jS], we define a 
normal series for a group G to be a chain of subgroups 1 = Gq<Gi< - ■ -<G C = G 
with the additional condition that each Gi < G. 

Theorem 2.2. Suppose a group G has a finite normal series 



in which each quotient Gj+x/Gj is cyclic or is contained in Z(G/Gi). Then 
D(G) < c+1. 

Proof. We proceed by induction on c. If c = 0, then G = Go = 1 and D(G) = 1. 
Now let G have a normal series 1 = Go < G\ < • • • < G c = G of length c > with 
the required property. In order to apply Lemma 12. II we wish to show that any 
two conjugate elements lying in Gi generate the same subgroup, and in addition 
any subgroup of the quotient group G/Gi acts with distinguishing number at 
most c. 

Let ni and be conjugate elements in G\. We have assumed that G\ is 
either cyclic or contained in Z(G). In the former case, note that since conjuga- 
tion is a group automorphism, [G\ : (ni)] = [G\ : (^2)]- But G\ is cyclic, so it 
has precisely one subgroup of this index. Therefore (n\) — (71.2). In the latter 
case, note that every element of Z{G) has no conjugates other than itself, so 
n\ = n 2 and (m) = (n 2 ). 

Next, it follows from the Third Isomorphism Theorem that 



is a normal series for G/G\ of length c — 1 in which each quotient group 
(G 4+ i/Gi)/(G ?; /Gi)) is cyclic or is contained in Z{{G/G x )/{G i /G 1 )). Now for 
any subgroup L of G/Gi, let Li = (G;+i/Gi) n L for each i with < i < c— 1. 
Then one can check that 




D(G) < c+1. 



□ 



1 = G < Gi < ■ ■ ■ < G c = G 



1 = Gx/d < G2/G1 < • • • < G/G 



1 = Lq < L\ < ■ • ■ < L c _i = L 
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is a normal series of length c — 1 for L with the property that each quotient 
group Li+i/Li is cyclic or is contained in Z(L/Lj). Then D(L) < c by the 
inductive hypothesis. Thus, all the conditions of Lemma 12. II are satisfied, so 
D(G)<c+l. □ 

As consequences of Thcorcm l2.2l we obtain upper bounds on the distinguish- 
ing number of nilpotcnt and supersolvable groups. We recall the definitions of 
these important classes of groups below; see |S] for a more detailed discussion 
of them. 

A group G is said to be nilpotent if it possesses a finite normal series 1 = 
Go < G\ < • • • < G c = G such that each quotient group Gj + i/Gj is contained in 
Z{G/Gi). If the shortest such normal series has length c, then we say that G is 
nilpotent of class c. 

Corollary 2.3. Let G be nilpotent of class c. Then D(G) < c + 1. 

In particular, since all Abelian groups are class-1 nilpotcnt, we have D(G) < 

2 for G Abelian, as shown in 

A group G is said to be supersolvable if it possesses a finite normal series 
1 = Go < Gi < • • • < G c = G such that each quotient group Gj+i/Gj is cyclic. In 
this case, we will say that G is supersolvable of length c. See |3j for a detailed 
discussion of supersolvable groups. 

Corollary 2.4. Let G be supersolvable of length c. Then D(G) < c + 1. 

A group G is called metacyclic if it has a normal subgroup N < G such that 
both N and G/N are cyclic. Such groups have been completely classified in [5], 
and include all groups of squarefree order. 

Corollary 2.5. Let G be a metacyclic group. Then D(G) < 3. 

We obtain as a special case that if G is dihedral then D(G) < 3, as shown 

3 The action of GL n (K) on K n 

In this section, we consider the action of GL n (K) , the group ofnxn invertible 
matrices over a field K, on K n , the n-dimensional vector space over K. We 
may regard the elements of K n as column vectors and accordingly define a left 
action of GL n (K) on K n as A: v i-> Av for each v £ K n , A £ GL n {K). This 
action is clearly faithful. 

Our main result is that if K is sufficiently large, then 2 colors suffice to 
distinguish this action. 

Theorem 3.1. Let K be a field. If K is infinite or is finite of order greater 
than n + 1, then D GL r K \(K n ) = 2. 



5 



Proof. We first observe that the multiplicative group K x must contain a nonzero 
element a of order greater than n. For if K is infinite, then we may certainly 
choose such an a since there exist only finitely many solutions in K to the 
equations x l = 1 for each 1 < I < n. On the other hand, if K is finite, then we 
know that K x is a cyclic group of order | K\ — 1. Let a generate the group K x , 
then the order of a is \K\ — 1 > n. 

Now let ei, . . . , e„ be the standard basis vectors in if", and let S 1 be the 
set of vectors {a l ej < i < j < n}. Each of these vectors is distinct since a 
has order greater than n, so the cardinality of S is \n[n + 1). Now color every 
vector in S blue and all remaining vectors red. We claim this is a distinguishing 
2-coloring of K n with respect to the action of GL n {K). 

Suppose A G GL n {K) preserves this coloring. It suffices to show that Aek = 
ek for each efc. Since efc is blue, the image of must also be blue and so 
must have the form a l ej for < i < j < n. We wish to show that i = and 
j = k. First, note that Aek = a l ej implies that A~ 1 (a 1 ~ 1 ej) = a~ 1 A~ 1 (a l ej) = 
a~ 1 ek- Now, a~ 1 ek is a red point, because if instead a~ 1 ek = a c ek for some 

< c < k, then a would have order at most c + 1, but c + 1 < k < n and we 
assumed that the order of a was greater than n. So, a~ 1 tk is red, and since A 
was assumed to be color preserving, a t ~ 1 ej is also red. This is only possible if 

1 = 0. Thus Aek = e j and so A induces some permutation of the basis vectors 
{ei, . . . ,e„}. Suppose for a contradiction that A permutes them nontrivially. 
Then there must exist Aek = with k < j. Then a^ 1 ek is red since k < j — 1, 
but a^ 1 ej is blue, and A(a- 7_1 efc) = a-' _1 ej, a contradiction. Therefore, A 
fixes each basis vector ek and so A = 1„ as desired. We have exhibited a 
distinguishing 2-coloring of K n , so D GLn ^ K ){K n ) < 2. Now, it is possible that 
D GLn ( K ){K n ) = 1 only if GL n (K) is the trivial group. This occurs only when 
n = 1 and K = ¥2, which was excluded by the assumption that \K\ > n + 1. 
Therefore, we have the equality D GLn ^ K )(K n ) = 2. 

□ 

Thcorem l3.1l leaves open the case when the size of the field is relatively small. 
It is possible to show by case analysis that D GL2 i^ 2 ^{¥\) = -Dgl 2 (f 3 ) (^3) = 3. 
However, we leave the case when n > 2 and \K\ < n + 1 as an open problem. 

4 Discussion and open questions 

The distinguishing number seems to be a very natural property of group actions, 
and efforts to relate the distinguishing number of a group action to group prop- 
erties seem likely to be fruitful. Below, we give several possibilities for further 
investigation. 

One interesting method of attack relics on the following simple fact. 

Observation. Let G act faithfully on X . Fix a coloring c of X and let H c = 
{g S G I g preserves c}. Then H c is a subgroup of G. 

The distinguishing number, then, is the smallest number of colors admitting 
a coloring c such that H c — 1. Thus, it seems plausible that one could make 
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direct use of information on the subgroup structure of G to characterize the dis- 
tinguishing number. We present the theorem below as an example of employing 
this technique. 

Theorem 4.1. Let G be a finite group acting faithfully on a set X. Letp be the 
smallest prime dividing the order of G, and let M be the length of the largest 
orbit of the action of G on X. Then Dq(X) < 

Proof. With f^j-l colors, we may color each orbit of X such that every color 
class within a given orbit has size at most p — 1. Call this coloring c. Let ~ be 
the equivalence relation given by x\ ~ X2 if and only if x\ and x-i are in the same 
orbit and have the same color. Let Pi, P2, ■ ■ ■ , Pk be the equivalence classes of 
this relation, and let ni = \Pi\. Note that a color preserving permutation h G H c 
can take a given element only to another element in its equivalence class. Thus 
H c < S ni x S n2 x • • • x S nk . Also, H c < G. But since each rij < p, the orders of 
S ni x S 7l2 x • • • x S„ k and G are relatively prime. Since the order of H c divides 
both orders, we have H c = 1, and so c is distinguishing. □ 

In addition, we could use subgroup structure as a way to generalize the notion 
of the distinguishing number, as follows. Given a group G acting faithfully on 
a set X and H a subgroup of G, let Dg,h(X) denote the smallest number of 
colors admitting a coloring of X such that the only elements of G that induce 
color-preserving permutations lie in H. Thus, when H = 1, we recover the 
original notion of the distinguishing number. 

Question 1. Characterize Dg,h(X). 

In Sectional we considered the maximum distinguishing number admitted 
by a given group. Intuitively, we would expect a large group to admit actions 
that require many colors to distinguish them. Thus, we ask whether the distin- 
guishing number is ordered in a way that respects the partial ordering of groups 
defined by subgroup inclusion. 

Question 2. Let G and H be groups, H a subgroup of G. Does it follow that 
D{H) < D(G) ? 

Note that if a given faithful action of H on X can be extended to a faithful 
action of G on X then Dn(X) < Dq(X), because any coloring of X that is 
distinguishing with respect to the action of G is also distinguishing with respect 
to the action of H . However, since not every faithful action of H on X can 
necessarily be extended to a faithful action of G (for example if \G\ > |X|!), the 
question cannot be answered immediately in the affirmative. 

We also ask whether the bounds obtained in Section for nilpotcnt and 
supcrsolvable groups arc tight. 

Question 3. For which k > 2 does there exist a group G that is nilpotent group 
of class k (or a supersolvable group of length k ) acting faithfully on a set X such 
that D G (X) = k + l? 
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InSectionEl we showed that D GLn {K) (K n ) = 2if|#| > n+1 and D GLn[K) (K n ) = 
3 if \K\ G {2,3} and n = 2. As mentioned, we leave the remaining cases as an 
open question. 

Question 4. Compute DQ Ln ^ K ^{K n ) for n > 2 and \K\ < n + 1. 

The generalization of the distinguishing number to infinite groups acting 
on infinite sets is new, and it might be interesting to investigate conditions on 
the finiteness or infiniteness of the distinguishing number. This leads to many 
questions, including the following. 

Question 5. Suppose G is a group that always acts with finite distinguishing 
number. Does it follow that D{G) < co, that is, that the set {Dg(X)} has a 
maximum element? 

Another approach would simply be to define the distinguishing number of an 
action to be the cardinality of the smallest set of colors admitting a distinguish- 
ing coloring. This would eliminate the formal distinction between the finite and 
infinite cases. 

Finally, throughout this paper, we have considered the distinguishing num- 
bers that a fixed group admits in its actions on various sets. We could instead 
fix a set [n] and consider the distinguishing numbers it admits under the actions 
of various groups. 

Question 6. For each n, characterize the set 

T n = {Dcdn]) | G is a transitive subgroup of S n }. 

One may show that T n = {2, . . . , n} for n = 2, 3, 4, 5, and 6. We ask whether 
T n has this form for larger n. Note that we require our group G to be transitive, 
for otherwise each distinguishing number k between 1 and n could be achieved 
by taking a subgroup of S n that fixes each k + 1, k + 2, . . . , n and whose action 
on 1, . . . , k is isomorphic to the action of 
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